Compositional generalization, the ability of an agent to generalize to unseen combinations of latent factors, is easy for humans but hard for deep neural networks. A line of research in cognitive science has hypothesized a process, "iterated learning,"

Human reasoning can be understood as a cooperation between the intuitive, associative "System-1" and the deliberative, logical "System-2".

LLMs show remarkable emergent abilities, such as inferring concepts from presumably out-of-distribution prompts, known as in-context learning.

Modern deep neural networks, while strong in many domains [29], have notmastered comparable language-basedgeneralization challenges, afactconjectured tounderlie their sample inefficiencyand inflexibility [26,25,8].

Compositional generalization is a basic and essential intellective capability of human beings, which allows us to recombine known parts readily.

CFQ is the only realistic benchmark that comprehensively measure compositional generalization.